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ABSTRACT 


The scientific, social, and economic advantages that accrue from Open Science (OS) practices—ways of 
doing research that emphasize reproducibility, transparency, and accessibility at all stages of the research 
cycle—are now widely recognized in nations around the world and by international bodies such as the 
United Nations and the Organization for Economic Cooperation and Development. However, program wide 
or coordinated instruction of undergraduate students in OS practices remains uncommon. At the University 
of British Columbia in Canada, we have started to develop a comprehensive undergraduate OS program that 
can be adapted to and woven into diverse subject curricula. We report on the context and planning of the 
pilot module of the program, “Open Science 101”, its implementation in first-year Biology in Fall 2019, and 
qualitative results of an attitudinal survey of students following their course. 


1. INTRODUCTION 


In the Spring of 2018, the Library and the Department of Biology at the University of British Columbia’s 
(UBC’s) Okanagan Campus in Kelowna, Canada, embarked upon an undergraduate Open Science (OS) 
instructional program—a venture unique not only in Canada, but also in North America, as far as we are 
aware. This instructional program has since evolved into one wing of a larger Fostering OS initiative 
spanning all of UBC. 
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The initiative and growth of the undergraduate instructional program was made possible through the 
efforts of a number of champions at UBC who succeeded in achieving recognition of OS instruction as a 
strategic institutional priority. The program has seen planning, aided by relationships between the Library 
and the Department of Biology, for a multi-module program to be integrated into key courses across all 
four years of the undergraduate Biology degree. Starting in the fall of 2018, approximately 60 students in 
two third-year biology classes tested a pilot module to evaluate interest and means of implementing content 
that supported, without altering, existing curricula and class assignments. After this initial success and the 
subsequent hiring of a dedicated OS Librarian to support this work in the Spring of 2019, 328 students 
participated in a voluntary pilot module called Open Science 101 (OS 101) in two first-year Biology classes 
in Fall 2019. 


OS 101 defines OS as a movement to make products of all stages of the research lifecycle (e.g., scientific 
research, data, and publications) more accessible and provide transparency, openness, integrity, and 
reproducibility to scientific studies. The module provides an overview of the practical challenges of 
conducting reproducible research, the societal impact of irreproducible research, and philosophical and 
ethical issues surrounding OS. Assessment of the module revealed that a major attitudinal shift had occurred 
among participating students: They were convinced of the importance of a shift toward OS. We will look 
at the background and context surrounding our OS instruction; the planning, delivery, and evaluation of 
OS 101; and future plans for the undergraduate program. 


2. BACKGROUND: THE STATE OF OS IN CANADA 


For readers to understand the novelty of the UBC Okanagan program fully, we must situate this program 
within the Canadian context. Although Canada has lagged behind Europe in its promotion of OS, between 
2015 and 2020 the Canadian government introduced measures to promote Open Access [1], articulated a 
strategic plan to foster OS practices among federal government researchers and recipients of federal 
government funding [2], and participated in a UNESCO recommendation on the development of an 
inclusive OS policy framework [3]. These developments affected university researchers funded by Canada’s 
TriAgency, three government bodies that support scholarly research in Canada: the National Science and 
Engineering Council of Canada (NSERC), the Canadian Institutes of Health Research (CIHR, which had 
implemented an open access policy at an early date), and the Social Sciences and Humanities Research 
Council of Canada (SSHRC). 


These important efforts, preceded by similar initiatives in the United States and Europe, emphasize 
infrastructure, funding, and other supports targeted largely to the public sector and to practicing researchers 
or graduate students. But OS requires a culture of change that reaches beyond these domains, one that 
prepares future citizens and future researchers. It is with this understanding that we undertook our 
instructional program. 


A recent environmental scan confirmed that no other Canadian universities are delivering undergraduate 
instruction in OS [4]. Open scholarship instruction in Canadian post-secondary institutions appears to 
concentrate on three areas: 
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e Open Educational Resources (OER) as learning tools for students; 
e Open Access databases; 
e Research Data Management (primarily for graduate students). 


Although we found no evidence that any other universities were considering an undergraduate OS 
program on the scale of UBC’s, we know anecdotally that many individual instructors at a variety of 
universities are integrating aspects of OS into their labs and classrooms. At the very least, this suggests an 
appetite for a shift in how we teach the scientists of the future. 


This landscape, while perhaps more integrated at the institutional policy level, appears very much 
mirrored in the European Union. While there is little evidence of the degree of undergraduate integration 
we are attempting at UBC Okanagan, Horizon Europe, the 9th EU framework program for research and 
innovation, specifically names OS as one of its three pillars [5]. In accordance with the new program, many 
European universities are beginning to put comprehensive OS policies in place. One example of this is the 
Delft University of Technology (TU Delft) in the Netherlands. The TU Delft Strategic Plan OS: 2020-2024 
outlines plans to implement coordinated OS training modules for instructors, research staff, and students 
[6]. However, education for students at the undergraduate level is not mentioned. 


Given the nascent nature of the OS movement, scant literature on best practices in library OS information 
literacy instruction exists. Extant publications on this topic (e.g., [7]) tend to focus on information literacy 
instruction of researchers pursuing graduate studies—that is, EU Level 2 or higher. 
Some exceptions include Ayris and Ignat [8], who suggested involving undergraduates (EU Level 1 students) 
in Citizen Science projects; Button [9], who established an undergraduate Psychology consortium in Britain 
to introduce rigorous research practices to writers of undergraduate theses, and Jekel et al. [10], who 
incorporated replication projects into Honors thesis requirements. 


3. A SHORT HISTORY OF OS ADVOCACY AT UBC 


Efforts to promote best practices in OS at UBC began in earnest in 2015, under the direction of an 
assistant professor in the Biology Department of UBC Okanagan. Over the first two years efforts were largely 
focused on (i) identifying and reaching out to the administrative units that would be key to establishing OS 
as a core tenet of UBC's research agenda, and (ii) building a grassroots network of OS advocates. 
Key partners in this early work included the Vice-Provost and Association Vice President Academic of 
UBC Vancouver; UBC Okanagan’s Associate Vice-Principal and Associate Vice-President, Research and 
Innovation; and UBC Okanagan’s Chief Librarian and Associate Provost, Learning Services. As well, the 
movement was supported by UBC’s Advanced Research Computing group (ARC), who recognized at the 
outset how OS best practices aligned with their core mandate of facilitating research through high- 
performance computing infrastructure and support. 


Despite support from upper administrative positions and ARC, one challenge remained: All of this work 
was happening “off the sides of desks”, and so progress was at the mercy of already full work schedules. 
Nevertheless, between 2017 and 2019, a new Southern Medical Program Librarian, in conjunction with 
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ARC and the aforementioned Biology professor, succeeded in launching Canada’s first instance of OS 
Framework Institutions at UBC®. 


In 2019, the UBC Okanagan OS advocates secured $100K (CAD) in strategic funding for OS over two 
years. Their proposal included short- and long-term objectives within both the teaching and research realms, 
the former being described below under Sections 4 and 5. And the funds supported the hiring of a part-time 
OS Librarian who could focus on building the OS instructional program, and enable team members to 
broaden the reach of instructional seminars and workshops. 


4. UNDERGRADUATE SCIENCE EDUCATION 


Why target undergraduates? Johannes Vogel, director general of the Museum für Naturkunde and Professor 
of Biodiversity and Public Science at Humboldt University in Berlin, recognizes the power that young 
people have to mobilize movements and suggests that they can help to open up science. In his article 
“Scientists need to learn from the young”, Johannes Vogel states, 


Science must learn to listen, open up and again become part of the community. Such a transformation 
... will cost time and money, including the restructuring of the incentive systems in science itself [11]. 


Students—our future scientists, professors, practitioners, and policy makers—have long been catalysts of 
social and political change. As such, the transformation of academic culture we desire requires cultivating 
in our undergraduates an appreciation of OS values and familiarity with transparent and reproducible 
practices. 


Introducing OS values and practices as early as possible has numerous benefits, both for the prospective 
researchers and for the entire undergraduate population. First, by participating in OS training in the first 
year, students are better equipped to critically evaluate the materials and methods they are exposed to in 
subsequent years. Our qualitative experience is that students feel empowered by the early OS training, and 
therefore show greater interest and engagement when presented, for example, with case studies or primary 
research articles. We are hopeful that this applies more broadly, outside the classroom, where students need 
to make informed decisions as active members of society. 


Second, values and practices that are introduced early can be more fully developed and reinforced 
throughout the undergraduate curriculum, and this in turn increases the likelihood that the practices become 
habits. This is especially beneficial to students who aim to pursue science careers. Crucially, we have seen 
how these habits can propagate up through research labs to principal investigators, and ultimately precipitate 
wholesale changes to research workflows. 


Lastly, by their final year, students will be well equipped to apply OS best practices to research endeavors 
such as honors research projects and capstone projects. We therefore anticipate substantive improvements 
in the quality of undergraduate-led research, which is a focus of many undergraduate programs. 


© see https://osf.io/institutions/ubc/ 
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4.1 Undergraduate Program Vision 


Our initiative at the undergraduate level is two-fold. The first is to build a series of learning objects that 
will allow faculty and lab managers to adapt current curricula and assignments to a model that integrates 
the fundamental principles and practices of OS in the instruction and evaluation of their existing teaching 
practices. The goal here is not to change what is being taught, but how undergraduate students are taught 
and evaluated. The second is to build a test bed of integration working closely with instructors in the Biology 
Department at UBC’s Okanagan campus. This integration allows us to identify where discipline modularity 
in these objects is best situated and at what levels it makes the most sense to introduce various concepts 
related to OS. This in turn is allowing us to build better learning objects that can be adapted for different 
disciplines. 


4.2 Proposed Learning Objects 


At the outset, our undergraduate program sought to give students a grounding in several categories of 
best practices that facilitate reproducibility, replicability, and inclusiveness in science. These include open 
access, transparency at all stages of the research cycle, data management, citizen engagement, and 
applications of technology to OS. The program is designed to scaffold from introductory courses in first and 
second year to more advanced courses in third year; it will culminate in a fourth-year replication project. 
We intend the module templates to be flexible enough to integrate into courses in a variety of disciplines. 


The plan, and challenge, is to integrate this education smoothly into the existing course content in such 
a way that it becomes just another part of the experience of completing a science degree, and minimizes 
the burden to instructors. 


Our OS library information literacy program for undergraduates aims to provide templates and resources 
to enable instructors to weave OS principles and practices into the existing curriculum in such a way that 
Open practices become as automatic to students as putting on a lab coat. 


In creating a comprehensive undergraduate OS program that can be implemented in diverse disciplinary 
settings, we must consider that professors and instructors have varying levels of comfort with and knowledge 
about OS and its associated technologies and that OS practices look different in different subject contexts. 
Therefore, we plan to introduce some initial modules and later, assuming good results, open a dialog with 
faculty at large about our ideas for a comprehensive program. In the list below, italics indicate a module 
that has been implemented; as aspects of our work are released, they will be found on our website, https:// 
openscience.ubc.ca/. 


First Year 

e OS 101: Principles of OS (Definition, Rationale, Values, Benefits, and Constraints) 

e OS 102: Reproducibility through the Research Cycle; critical evaluation of scientific papers; basic 
data management 
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Second Year 

e OS 201: Reproducible analysis and literate programming using tools such as RMarkdown 

e OS 202: Modular Lab Components: Review of labs in stages (similar to the preregistration process) 
e OS 203: Digitization of lab books and field notes, & intro to electronic lab books 


Third Year 

e OS 301: Use and Creation of Open Data Collections with regard for security and privacy constraints 
e OS 302: Science communication, Citizen Science, or Participatory Research 

e OS 303: Peer review 

e OS 304: Advanced prevention of Questionable Research Practices 


Fourth Year 
e OS 401: Replication Project (optional, at the Department's discretion) 


5. IMPLEMENTATION OF OS 101 IN BIOLOGY 
5.1 Target Classes 


Since the Associate Professor in Biology had cultivated close relationships with his colleagues in Biology 
and had planted the seeds of change by speaking to them of the benefits of OS practices, this department 
seemed a natural setting in which to introduce our pilot project. OS 101 was to provide a general introduction 
to OS concepts and fits naturally into first year. Moreover, the lab manager for first-year Biology showed 
receptivity to the idea. Thus, the team built a module to be implemented in the Lab components of Biology 
116 (for Biology majors) and Biology 122 (for majors in other subjects). 


5.2 Objectives 


Our pilot was designed to give students an overview of the practical challenges of conducting reproducible 
research, the societal impact of irreproducible research, and philosophical and ethical issues surrounding 
OS. 


Our intent was for students to be able to explain the following on completion of the unit: 


e The nature of scientific knowledge 

e The science-society relationship 

e The replication crisis 

e Core values of OS 

e Benefits of OS 

e Barriers to the realization of OS ideals 
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5.3 Considerations 
5.3.1 Lightening the Load 


Both students and instructors face considerable demands on their time. To avoid creating extra work for 
instructors, we introduced OS 101 in the lab component of the course, rather than the lecture component, 
and made it a self-contained module with short unit quizzes that were marked automatically. Assessments 
at the start and end of OS 101 contained some open questions, but these were to be marked by the OS 
Librarian, not by the lab manager or teaching assistants. 


To avoid burdening the students overly, the lab manager and the OS Librarian arranged for the students 
to receive extra participation marks for attempting the quizzes; students were not penalized if they did not 
participate in the module. We chose a medium in which the module could be completed online and at 


the student's own pace: we mounted it on Canvas, UBC’s Learning Management System 
(see Figure 1). 

= SNBX Open Science > Pages > A Replication Crisis 

Home View All Pages S Publish X Edit 


Announcements © 


Assignments 


A Replication Crisis 


Discussions 
S Introduction 
People 

Open Science addresses many problems with how research is done. At its core, it aims to 
Pages ensure that research can be confirmed through replication, and that both new 
Files discoveries and replications of previous research get rewarded in a similar way. 
— Replication 
Outcomes @ 

Although scientific progress rests on the repetition and reproduction of results, efforts 
Rubrics at this are hindered by two main factors: 
Quizzes e First, there is a desire for the new and shiny; that is, novel studies are more 
Modules interesting. 

e Second, many studies cannot be competently analyzed or replicated. This is because 

My Media critical information about them — design, data, methods, lab notes, analyses and code 


Media Gallery 


— may not be made available. 


Figure 1. Screenshot of OS 101 Canvas content (unpublished, Fall 2019). 
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5.3.2 Relatability 


We tried to provide examples that the students could relate to based on their own experience: A case 
in point is this example relating the desirability of transparency in practices to a student's wish to know 
how and why marks are assigned for an assignment: 


Transparency and Grading 

One case where students want and need transparency is in grading. Most of you probably appreciate 
knowing how marks were given to questions on a midterm exam and what criteria the marker used 
to score each question. This provides you with both a reason for the assigned grade and a means of 
comparing your grade with those of your classmates. Ultimately, this specific information can help 
you to address gaps in your knowledge and perform better on future assessments (such as the final 
exam). In the same way, transparency in research allows scientists to improve future studies and add 
to accumulated knowledge [12]. 


We also introduced dialogs to make the content feel more informal or “low-key”. 


5.3.3 Accessibility 


In making the material accessible to first-year students, it was imperative to consider their limited 
familiarity with (a) research data collection, management, and sharing and (b) statistics. Note that first-year 
students, the target of this module, are not yet engaged in collecting or sharing their own data. As such, 
we did not refer specifically to initiatives such as FAIR data principles [13]. However, data are mentioned 
as being critical for reproducibility and replicability, and later modules will discuss FAIR data principles. 


In a similar vein, we had to consider that most students would not have taken a university course in 
statistics at this point in their careers. Consequently, when mentioning questionable research practices, we 
did not refer to topics such as p-values or margins of error: We kept our explanations more general. For 
instance, we defined replication as “thorough repetition” and skirted the issue of statistical significance in 
our exploration of publication bias and questionable research practices. 


Finally, we could not take for granted that the students coming into first year remembered learning about 
the nature of scientific knowledge and the scientific method, or had in fact received such instruction in 
high school. Students come from various educational backgrounds. Therefore, in the first unit, we laid a 
foundation for the introduction of OS by first examining how scientific knowledge is built. 


5.3.4 Assessment and Feedback 


At the end of the term, students had the option to answer open-ended questions through an exit survey 
mounted on Canvas. We posed the following questions: 


e If you had to sum up your view on OS, what would you say? 
e In your opinion, what is the most important benefit that OS can bring? 
e Which aspects of the OS movement discussed this semester resonate the most with you? 
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As noted in our presentation at the OSC2020 Conference in Berlin in March, the students showed a keen 
interest in OS principles, and professed, by the end of the course, both positive attitudes toward OS 
practices and an appreciation of the ways in which OS can improve science and society. The top benefits 
of OS as cited by the students were scientific progress, transparency, and collaboration. 


Seventy-two students responded to the survey, and many of them wrote inspiring and moving words, 
demonstrating that the module had indeed engendered in them an appreciation of OS as a positive 
movement for scientific research and for society as a whole [14, 15]. The following are two examples of 
the feedback [14, 15]. 


I think OS principles are very important, and can provide a good foundation for future science 
projects. I think it is important that as students entering the scientific world, that we learn about OS 
now so we can carry these principles with us through our careers. 


[OS] is something that goes against the mainstream competitive nature of the science community, but 
it is mutually beneficial for all scientists and society, so | support it fully. 


Below is a “wordle” created from the text of students’ survey responses (Figure 2): 


Figure 2. Image created at https:/www.wordclouds.com/ by Sharon Hanna in February 2020 from the text of 
attitudinal survey responses. 


In the last year and half working on this project, we have been struck by the value of the combined 
support provided to this initiative by faculty, the Library, the Provost's office, our Advanced Research 
Computing Department, and the Strategic Initiatives and Planning office at UBC. While the Library has 
taken the lead on the promotion and development of these undergraduate materials, lacking this support 
from any one of these bodies would have made our efforts much more challenging. 
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6. NEXT STEPS AND CONCLUSION 


Although we created detailed content for OS 101 and 102, our role in OS instruction is changing to one 
of providing templates, resources, and consulting advice to instructors and lab managers interested in 
introducing OS principles and practices into their classrooms and labs. As such, we’ve been introducing 
alternative examples that speak to course areas other than Biology and examples related to COVID-19. 
Producing templates for second-year modules lies just ahead on the horizon. 


The ability to make discrete connections across the research life cycle and to critically identify sources 
of bias underpins OS principles and practices. Working with undergraduate students and liaising with 
faculty and lab managers, our OS Librarian introduces students to this landscape of scholarly communication 
in the context of OS. 


In the summer and fall of 2021, we will be working with the Biology Lab Managers to tie OS 101 and 
102 more closely to the curriculum and to introduce lab exercises that specifically link to the content. Lab 
teaching assistants will also receive an OS orientation. OS will make up a larger portion of a student's 
overall course mark. 


At this point, whether the current funding of an OS Librarian position through the President’s Excellence 
fund will be renewed remains unknown. The COVID-19 pandemic has affected our project both by 
introducing uncertainty into the university’s budget forecast and by raising awareness of the boons that OS 
practices such as data sharing and open-source medical equipment designs can bring. Our qualitative 
results show that it is possible to foster change in the next generation of scientists with a relatively small 
investment of human resources, and given the policy changes occurring in the Canadian government and 
the international context, there seems little doubt that OS will continue to grow in importance. 
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